ebab7d0da53c51438720d60c6e2e467c - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>Random testing</title>

</head>

Random testing

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .ambox{border:1px solid #a2a9b1;border-left:10px solid #36c;background-color:#fbfbfb;box-sizing:border-box}.mw-parser-output .ambox+link+.ambox,.mw-parser-output .ambox+link+style+.ambox,.mw-parser-output .ambox+link+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+style+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+link+.ambox{margin-top:-1px}html body.mediawiki .mw-parser-output .ambox.mbox-small-left{margin:4px 1em 4px 0;overflow:hidden;width:238px;border-collapse:collapse;font-size:88%;line-height:1.25em}.mw-parser-output .ambox-speedy{border-left:10px solid #b32424;background-color:#fee7e6}.mw-parser-output .ambox-delete{border-left:10px solid #b32424}.mw-parser-output .ambox-content{border-left:10px solid #f28500}.mw-parser-output .ambox-style{border-left:10px solid #fc3}.mw-parser-output .ambox-move{border-left:10px solid #9932cc}.mw-parser-output .ambox-protection{border-left:10px solid #a2a9b1}.mw-parser-output .ambox .mbox-text{border:none;padding:0.25em 0.5em;width:100%}.mw-parser-output .ambox .mbox-image{border:none;padding:2px 0 2px 0.5em;text-align:center}.mw-parser-output .ambox .mbox-imageright{border:none;padding:2px 0.5em 2px 0;text-align:center}.mw-parser-output .ambox .mbox-empty-cell{border:none;padding:0;width:1px}.mw-parser-output .ambox .mbox-image-div{width:52px}@media(min-width:720px){.mw-parser-output .ambox{margin:0 10%}}@media print{body.ns-0 .mw-parser-output .ambox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style>

Random testing is a black-box <a href="Software_testing" title="Software testing">software testing</a> technique where programs are tested by <a href="Random_number_generation" title="Random number generation">generating</a> random, independent inputs. Results of the output are compared against software specifications to verify that the test output is pass or fail.<a href="#cite_note-Hamlet94-1">[1]</a> In case of absence of specifications the exceptions of the language are used which means if an exception arises during test execution then it means there is a fault in the program, it is also used as a way to avoid biased testing.

<div class="mw-heading mw-heading2"><h2 id="History_of_random_testing">History of random testing</h2></div>

Random testing for hardware was first examined by Melvin Breuer in 1971 and initial effort to evaluate its effectiveness was done by Pratima and <a href="Vishwani_Agrawal" title="Vishwani Agrawal">Vishwani Agrawal</a> in 1975.<a href="#cite_note-2">[2]</a>

In software, Duran and Ntafos had examined random testing in 1984.<a href="#cite_note-3">[3]</a>

The use of hypothesis testing as a theoretical basis for random testing was described by Howden in Functional Testing and Analysis. The book also contained the development of a simple formula for estimating the number of tests n that are needed to have confidence at least 1-1/n in a failure rate of no larger than 1/n. The formula is the lower bound nlogn, which indicates the large number of failure-free tests needed to have even modest confidence in a modest failure rate bound.<a href="#cite_note-:0-4">[4]</a>

<div class="mw-heading mw-heading2"><h2 id="Overview">Overview</h2></div>

Consider the following C++ function:

<div class="mw-highlight mw-highlight-lang-cpp mw-content-ltr" dir="ltr"><pre>int myAbs(int x) {

if (x > 0) {

return x;

}

else {

return x; // bug: should be '-x'

}

}

</pre></div>

Now the random tests for this function could be {123, 36, -35, 48, 0}. Only the value '-35' triggers the bug. If there is no reference implementation to check the result, the bug still could go unnoticed. However, an <a href="Assertion_(software_development)" title="Assertion (software development)">assertion</a> could be added to check the results, like:

<div class="mw-highlight mw-highlight-lang-cpp mw-content-ltr" dir="ltr"><pre>void testAbs(int n) {

for (int i=0; i<n; i++) {

int x = getRandomInput();

int result = myAbs(x);

assert(result >= 0);

}

}

</pre></div>

The reference implementation is sometimes available, e.g. when implementing a simple algorithm in a much more complex way for better performance. For example, to test an implementation of the <a href="Sch%C3%B6nhage%E2%80%93Strassen_algorithm" title="Schönhage–Strassen algorithm">Schönhage–Strassen algorithm</a>, the standard "*" operation on integers can be used:

<div class="mw-highlight mw-highlight-lang-cpp mw-content-ltr" dir="ltr"><pre>int getRandomInput() {

// …

}

void testFastMultiplication(int n) {

long x = getRandomInput();

long y = getRandomInput();

long result = fastMultiplication(x, y);

assert(x * y == result);

}

}

</pre></div>

While this example is limited to simple types (for which a simple random generator can be used), tools targeting object-oriented languages typically explore the program to test and find generators (constructors or methods returning objects of that type) and call them using random inputs (either themselves generated the same way or generated using a pseudo-random generator if possible). Such approaches then maintain a pool of randomly generated objects and use a probability for either reusing a generated object or creating a new one.<a href="#cite_note-AutoTest-5">[5]</a>

<div class="mw-heading mw-heading2"><h2 id="On_randomness">On randomness</h2></div>

According to the seminal paper on random testing by D. Hamlet

<blockquote>[..] the technical, mathematical meaning of "random testing" refers to an explicit lack of "system" in the choice of test data, so that there is no correlation among different tests.<a href="#cite_note-Hamlet94-1">[1]</a></blockquote>

<div class="mw-heading mw-heading2"><h2 id="Strengths_and_weaknesses">Strengths and weaknesses</h2></div>

Random testing is praised for the following strengths:

<ul><li>It is cheap to use: it does not need to be smart about the program under test.</li>

<li>It does not have any bias: unlike manual testing, it does not overlook bugs because there is misplaced trust in some code.</li>

<li>It is quick to find bug candidates: it typically takes a couple of minutes to perform a testing session.</li>

<li>If software is properly specified: it finds real bugs.</li></ul>

The following weaknesses have been described :

<ul><li>It only finds basic bugs (e.g. <a href="Null_pointer" title="Null pointer">null pointer</a> dereferencing).</li>

<li>It is only as precise as the specification and specifications are typically imprecise.</li>

<li>It compares poorly with other techniques to find bugs (e.g. <a href="Static_program_analysis" title="Static program analysis">static program analysis</a>).</li>

<li>If different inputs are randomly selected on each test run, this can create problems for <a href="Continuous_integration" title="Continuous integration">continuous integration</a> because the same tests will pass or fail randomly.<a href="#cite_note-so-6">[6]</a></li>

<li>Some argue that it would be better to thoughtfully cover all relevant cases with manually constructed tests in a white-box fashion, than to rely on randomness.<a href="#cite_note-so-6">[6]</a></li>

<li>It may require a very large number of tests for modest levels of confidence in modest failure rates. For example, it will require 459 failure-free tests to have at least 99% confidence that the probability of failure is less than 1/100.<a href="#cite_note-:0-4">[4]</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="Types_of_random_testing">Types of random testing</h2></div>

<div class="mw-heading mw-heading3"><h3 id="With_respect_to_the_input">With respect to the input</h3></div>

<ul><li>Random input sequence generation (i.e. a sequence of method calls)</li>

<li>Random sequence of data inputs (sometimes called stochastic testing) - e.g. a random sequence of method calls</li>

<li>Random data selection from existing database</li></ul>

<div class="mw-heading mw-heading3"><h3 id="Guided_vs._unguided">Guided vs. unguided</h3></div>

<ul><li>undirected random test generation - with no heuristics to guide its search</li>

<li>directed random test generation - e.g. "feedback-directed random test generation"<a href="#cite_note-PachecoLET2007-7">[7]</a> and "adaptive random testing" <a href="#cite_note-ART-8">[8]</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="Implementations">Implementations</h2></div>

Some tools implementing random testing:

<ul><li><a href="QuickCheck" title="QuickCheck">QuickCheck</a> - a famous test tool, originally developed for <a href="Haskell_(programming_language)" class="mw-redirect" title="Haskell (programming language)">Haskell</a> but ported to many other languages, that generates random sequences of API calls based on a model and verifies system properties that should hold true after each run.</li>

<li>Randoop - generates sequences of methods and constructor invocations for the classes under test and creates <a href="JUnit" title="JUnit">JUnit</a> tests from these</li>

<li>Simulant - a <a href="Clojure" title="Clojure">Clojure</a> tool that runs simulations of various agents (e.g. users with different behavioral profiles) based on a statistical model of their behavior, recording all the actions and results into a database for later exploration and verification</li>

<li>AutoTest - a tool integrated to EiffelStudio testing automatically Eiffel code with contracts based on the eponymous research prototype.<a href="#cite_note-AutoTest-5">[5]</a>·</li>

<li>York Extensible Testing Infrastructure (YETI) - a language agnostic tool which targets various programming languages (Java, JML, CoFoJa, .NET, C, Kermeta).</li>

<li>GramTest - a grammar based random testing tool written in Java, it uses BNF notation to specify input grammars.</li></ul>

<div class="mw-heading mw-heading2"><h2 id="Critique">Critique</h2></div>

<blockquote>Random testing has only a specialized niche in practice, mostly because an effective oracle is seldom available, but also because of difficulties with the operational profile and with generation of pseudorandom input values.<a href="#cite_note-Hamlet94-1">[1]</a></blockquote>

A <a href="Test_oracle" title="Test oracle">test oracle</a> is an instrument for verifying whether the outcomes match the program specification or not. An operation profile is knowledge about usage patterns of the program and thus which parts are more important.

For programming languages and platforms which have contracts (e.g. Eiffel. .NET or various extensions of Java like JML, CoFoJa...) contracts act as natural oracles and the approach has been applied successfully.<a href="#cite_note-AutoTest-5">[5]</a> In particular, random testing finds more bugs than manual inspections or user reports (albeit different ones).<a href="#cite_note-ManualvsRandom-9">[9]</a>

<ul><li><a href="Fuzz_testing" class="mw-redirect" title="Fuzz testing">Fuzz testing</a> - a kind of random testing which provides invalid input to the tested program</li>

<li><a href="Lazy_systematic_unit_testing#Systematic_Testing" title="Lazy systematic unit testing">Lazy systematic unit testing#Systematic Testing</a> - a systematic way of exploring "all" method calls, as implemented e.g. by NASA's <a rel="nofollow" class="external text" href="https://web.archive.org/web/20110107010258/http://babelfish.arc.nasa.gov/trac/jpf/wiki/intro/what_is_jpf">Java Path Finder</a> (which blends <a rel="nofollow" class="external text" href="https://web.archive.org/web/20110106040136/http://babelfish.arc.nasa.gov/trac/jpf/wiki/intro/testing_vs_model_checking">testing with model checking</a> by limiting the state space to a reasonable size by various means)</li>

<li><a href="SystemVerilog#Constrained_random_generation" title="SystemVerilog">Constrained random generation in SystemVerilog</a></li>

<li><a href="Corner_case" title="Corner case">Corner case</a></li>

<li><a href="Concolic_testing" title="Concolic testing">Concolic testing</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFRichard_Hamlet1994" class="citation book cs1">Richard Hamlet (1994). "Random Testing". In John J. Marciniak (ed.). Encyclopedia of Software Engineering (1st ed.). John Wiley and Sons. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0471540021</bdi>.</cite>

</li>

<li id="cite_note-2"><a href="#cite_ref-2">^</a> <cite id="CITEREFAgrawalAgrawal1975" class="citation journal cs1">Agrawal, P.; Agrawal, V. D. (1 July 1975). "Probabilistic Analysis of Random Test Generation Method for Irredundant Combinational Logic Networks". IEEE Transactions on Computers. C-24 (7): 691–695. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FT-C.1975.224289">10.1109/T-C.1975.224289</a>.</cite>

</li>

<li id="cite_note-3"><a href="#cite_ref-3">^</a> <cite id="CITEREFDuranNtafos1984" class="citation journal cs1 cs1-prop-long-vol">Duran, J. W.; Ntafos, S. C. (1 July 1984). "An Evaluation of Random Testing". IEEE Transactions on Software Engineering. SE-10 (4): 438–444. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FTSE.1984.5010257">10.1109/TSE.1984.5010257</a>.</cite>

</li>

<li id="cite_note-:0-4">^ <a href="#cite_ref-:0_4-0">a</a> <a href="#cite_ref-:0_4-1">b</a> <cite id="CITEREFHowden1987" class="citation book cs1">Howden, William (1987). Functional Program Testing and Analysis. New York: McGraw Hill. pp. 51–53. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>0-07-030550-1</bdi>.</cite>

</li>

<li id="cite_note-AutoTest-5">^ <a href="#cite_ref-AutoTest_5-0">a</a> <a href="#cite_ref-AutoTest_5-1">b</a> <a href="#cite_ref-AutoTest_5-2">c</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://se.inf.ethz.ch/research/autotest/">"AutoTest - Chair of Software Engineering"</a>. se.inf.ethz.ch. Retrieved 15 November 2017.</cite>

</li>

<li id="cite_note-so-6">^ <a href="#cite_ref-so_6-0">a</a> <a href="#cite_ref-so_6-1">b</a> <cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://stackoverflow.com/q/636353">"Is it a bad practice to randomly-generate test data?"</a>. stackoverflow.com. Retrieved 15 November 2017.</cite>

</li>

<li id="cite_note-PachecoLET2007-7"><a href="#cite_ref-PachecoLET2007_7-0">^</a> <cite id="CITEREFPachecoShuvendu_K._LahiriMichael_D._ErnstThomas_Ball2007" class="citation book cs1">Pacheco, Carlos; Shuvendu K. Lahiri; Michael D. Ernst; Thomas Ball (May 2007). <a rel="nofollow" class="external text" href="http://people.csail.mit.edu/cpacheco/publications/feedback-random.pdf">"Feedback-Directed Random Test Generation"</a> (PDF). 29th International Conference on Software Engineering (ICSE'07). pp. 75–84. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FICSE.2007.37">10.1109/ICSE.2007.37</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-7695-2828-1</bdi>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0270-5257">0270-5257</a>.</cite>

</li>

<li id="cite_note-ART-8"><a href="#cite_ref-ART_8-0">^</a> <cite id="CITEREFT.Y._ChenF.-C._KuoR.G._MerkelT.H._Tse2010" class="citation cs2">T.Y. Chen; F.-C. Kuo; R.G. Merkel; T.H. Tse (2010), <a rel="nofollow" class="external text" href="https://figshare.com/articles/journal_contribution/26243711">"Adaptive random testing: The ART of test case diversity"</a>, Journal of Systems and Software, 83 (1): 60–66, <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.jss.2009.02.022">10.1016/j.jss.2009.02.022</a>, <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<a rel="nofollow" class="external text" href="https://hdl.handle.net/10722%2F89054">10722/89054</a></cite>

</li>

<li id="cite_note-ManualvsRandom-9"><a href="#cite_ref-ManualvsRandom_9-0">^</a> <cite id="CITEREFIlinca_CiupaAlexander_PretschnerManuel_OriolAndreas_Leitner2009" class="citation journal cs1">Ilinca Ciupa; Alexander Pretschner; Manuel Oriol; Andreas Leitner; Bertrand Meyer (2009). "On the number and nature of faults found by random testing". Software Testing, Verification and Reliability. 21: 3–28. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1002%2Fstvr.415">10.1002/stvr.415</a>.</cite>

</li>

</ol></div></div>

<div class="mw-heading mw-heading2"><h2 id="External_links">External links</h2></div>

<ul><li><a rel="nofollow" class="external text" href="http://www.uio.no/studier/emner/matnat/ifi/INF4290/v10/undervisningsmateriale/INF4290-RandomTesting.pdf">Random testing</a> by Andrea Arcuri.</li>

<li><a rel="nofollow" class="external text" href="http://web.cecs.pdx.edu/~hamlet/random.pdf">Random testing</a> by Richard Hamlet, professor emeritus at Portland State University; a valuable list of resources at the end of the paper</li>

<li><a rel="nofollow" class="external text" href="http://c2.com/cgi/wiki?RandomTesting">Random Testing wiki</a> at Cunningham & Cunningham, Inc.</li></ul>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hlist dl,.mw-parser-output .hlist ol,.mw-parser-output .hlist ul{margin:0;padding:0}.mw-parser-output .hlist dd,.mw-parser-output .hlist dt,.mw-parser-output .hlist li{margin:0;display:inline}.mw-parser-output .hlist.inline,.mw-parser-output .hlist.inline dl,.mw-parser-output .hlist.inline ol,.mw-parser-output .hlist.inline ul,.mw-parser-output .hlist dl dl,.mw-parser-output .hlist dl ol,.mw-parser-output .hlist dl ul,.mw-parser-output .hlist ol dl,.mw-parser-output .hlist ol ol,.mw-parser-output .hlist ol ul,.mw-parser-output .hlist ul dl,.mw-parser-output .hlist ul ol,.mw-parser-output .hlist ul ul{display:inline}.mw-parser-output .hlist .mw-empty-li{display:none}.mw-parser-output .hlist dt::after{content:": "}.mw-parser-output .hlist dd::after,.mw-parser-output .hlist li::after{content:" · ";font-weight:bold}.mw-parser-output .hlist dd:last-child::after,.mw-parser-output .hlist dt:last-child::after,.mw-parser-output .hlist li:last-child::after{content:none}.mw-parser-output .hlist dd dd:first-child::before,.mw-parser-output .hlist dd dt:first-child::before,.mw-parser-output .hlist dd li:first-child::before,.mw-parser-output .hlist dt dd:first-child::before,.mw-parser-output .hlist dt dt:first-child::before,.mw-parser-output .hlist dt li:first-child::before,.mw-parser-output .hlist li dd:first-child::before,.mw-parser-output .hlist li dt:first-child::before,.mw-parser-output .hlist li li:first-child::before{content:" (";font-weight:normal}.mw-parser-output .hlist dd dd:last-child::after,.mw-parser-output .hlist dd dt:last-child::after,.mw-parser-output .hlist dd li:last-child::after,.mw-parser-output .hlist dt dd:last-child::after,.mw-parser-output .hlist dt dt:last-child::after,.mw-parser-output .hlist dt li:last-child::after,.mw-parser-output .hlist li dd:last-child::after,.mw-parser-output .hlist li dt:last-child::after,.mw-parser-output .hlist li li:last-child::after{content:")";font-weight:normal}.mw-parser-output .hlist ol{counter-reset:listitem}.mw-parser-output .hlist ol>li{counter-increment:listitem}.mw-parser-output .hlist ol>li::before{content:" "counter(listitem)"\a0 "}.mw-parser-output .hlist dd ol>li:first-child::before,.mw-parser-output .hlist dt ol>li:first-child::before,.mw-parser-output .hlist li ol>li:first-child::before{content:" ("counter(listitem)"\a0 "}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1236075235">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbox{box-sizing:border-box;border:1px solid #a2a9b1;width:100%;clear:both;font-size:88%;text-align:center;padding:1px;margin:1em auto 0}.mw-parser-output .navbox .navbox{margin-top:0}.mw-parser-output .navbox+.navbox,.mw-parser-output .navbox+.navbox-styles+.navbox{margin-top:-1px}.mw-parser-output .navbox-inner,.mw-parser-output .navbox-subgroup{width:100%}.mw-parser-output .navbox-group,.mw-parser-output .navbox-title,.mw-parser-output .navbox-abovebelow{padding:0.25em 1em;line-height:1.5em;text-align:center}.mw-parser-output .navbox-group{white-space:nowrap;text-align:right}.mw-parser-output .navbox,.mw-parser-output .navbox-subgroup{background-color:#fdfdfd}.mw-parser-output .navbox-list{line-height:1.5em;border-color:#fdfdfd}.mw-parser-output .navbox-list-with-group{text-align:left;border-left-width:2px;border-left-style:solid}.mw-parser-output tr+tr>.navbox-abovebelow,.mw-parser-output tr+tr>.navbox-group,.mw-parser-output tr+tr>.navbox-image,.mw-parser-output tr+tr>.navbox-list{border-top:2px solid #fdfdfd}.mw-parser-output .navbox-title{background-color:#ccf}.mw-parser-output .navbox-abovebelow,.mw-parser-output .navbox-group,.mw-parser-output .navbox-subgroup .navbox-title{background-color:#ddf}.mw-parser-output .navbox-subgroup .navbox-group,.mw-parser-output .navbox-subgroup .navbox-abovebelow{background-color:#e6e6ff}.mw-parser-output .navbox-even{background-color:#f7f7f7}.mw-parser-output .navbox-odd{background-color:transparent}.mw-parser-output .navbox .hlist td dl,.mw-parser-output .navbox .hlist td ol,.mw-parser-output .navbox .hlist td ul,.mw-parser-output .navbox td.hlist dl,.mw-parser-output .navbox td.hlist ol,.mw-parser-output .navbox td.hlist ul{padding:0.125em 0}.mw-parser-output .navbox .navbar{display:block;font-size:100%}.mw-parser-output .navbox-title .navbar{float:left;text-align:left;margin-right:0.5em}body.skin--responsive .mw-parser-output .navbox-image img{max-width:none!important}@media print{body.ns-0 .mw-parser-output .navbox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style></div><div role="navigation" class="navbox" aria-labelledby="Software_testing107" style="padding:3px"><table class="nowraplinks hlist mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><style data-mw-deduplicate="TemplateStyles:r1239400231">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}html.skin-theme-clientpref-night .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}@media(prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}}@media print{.mw-parser-output .navbar{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div id="Software_testing107" style="font-size:114%;margin:0 4em"><a href="Software_testing" title="Software testing">Software testing</a></div></th></tr><tr><th scope="row" class="navbox-group" style="width:1%">Test levels</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Acceptance_testing" title="Acceptance testing">Acceptance testing</a></li>

<li><a href="System_integration_testing" title="System integration testing">System integration testing</a></li>

<li><a href="System_testing" title="System testing">System testing</a></li>

<li><a href="Integration_testing" title="Integration testing">Integration testing</a></li>

<li><a href="Unit_testing" title="Unit testing">Unit testing</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Test types, techniques, <a href="Software_testing_tactics" title="Software testing tactics">tactics</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><td colspan="2" class="navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="A/B_testing" title="A/B testing">A/B testing</a></li>

<li><a href="Benchmark_(computing)" title="Benchmark (computing)">Benchmark</a></li>

<li><a href="Compatibility_testing" title="Compatibility testing">Compatibility testing</a></li>

<li><a href="Concolic_testing" title="Concolic testing">Concolic testing</a></li>

<li><a href="Concurrent_testing" title="Concurrent testing">Concurrent testing</a></li>

<li><a href="Conformance_testing" title="Conformance testing">Conformance testing</a></li>

<li><a href="Continuous_testing" title="Continuous testing">Continuous testing</a></li>

<li><a href="Destructive_testing" title="Destructive testing">Destructive testing</a></li>

<li><a href="Development_testing" title="Development testing">Development testing</a></li>

<li><a href="Differential_testing" title="Differential testing">Differential testing</a></li>

<li><a href="Dynamic_program_analysis" title="Dynamic program analysis">Dynamic program analysis</a></li>

<li><a href="Installation_testing" title="Installation testing">Installation testing</a></li>

<li><a href="Negative_testing" title="Negative testing">Negative testing</a></li>

<li><a href="Regression_testing" title="Regression testing">Regression testing</a></li>

<li><a href="Security_testing" title="Security testing">Security testing</a></li>

<li><a href="Smoke_testing_(software)" title="Smoke testing (software)">Smoke testing (software)</a></li>

<li><a href="Software_performance_testing" title="Software performance testing">Software performance testing</a></li>

<li><a href="Stress_testing_(software)" title="Stress testing (software)">Stress testing</a></li>

<li><a href="Symbolic_execution" title="Symbolic execution">Symbolic execution</a></li>

<li><a href="Test_automation" title="Test automation">Test automation</a></li>

<li><a href="Usability_testing" title="Usability testing">Usability testing</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">[x]-box style</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Black-box_testing" title="Black-box testing">Black-box testing</a>

<ul><li><a href="All-pairs_testing" title="All-pairs testing">All-pairs testing</a></li>

<li><a href="Exploratory_testing" title="Exploratory testing">Exploratory testing</a></li>

<li><a href="Fuzz_testing" class="mw-redirect" title="Fuzz testing">Fuzz testing</a></li>

<li><a href="Model-based_testing" title="Model-based testing">Model-based testing</a></li>

<li><a href="Scenario_testing" title="Scenario testing">Scenario testing</a></li></ul></li>

<li><a href="Grey-box_testing" class="mw-redirect" title="Grey-box testing">Grey-box testing</a></li>

<li><a href="White-box_testing" title="White-box testing">White-box testing</a>

<ul><li><a href="API_testing" title="API testing">API testing</a></li>

<li><a href="Mutation_testing" title="Mutation testing">Mutation testing</a></li>

<li><a href="Static_testing" class="mw-redirect" title="Static testing">Static testing</a></li></ul></li></ul>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">See also</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Graphical_user_interface_testing" title="Graphical user interface testing">Graphical user interface testing</a></li>

<li><a href="Manual_testing" title="Manual testing">Manual testing</a></li>

<li><a href="Orthogonal_array_testing" title="Orthogonal array testing">Orthogonal array testing</a></li>

<li><a href="Pair_testing" title="Pair testing">Pair testing</a></li>

<li><a href="Soak_testing" title="Soak testing">Soak testing</a></li>

<li><a href="Software_reliability_testing" title="Software reliability testing">Software reliability testing</a></li>

<li><a href="Stress_testing" title="Stress testing">Stress testing</a></li>

<li><a href="Web_testing" title="Web testing">Web testing</a></li></ul>

</div></td></tr></tbody></table></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-02-09" href="https://en.wikipedia.org/wiki/?title=Random_testing&oldid=1274802838">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>